ALI Performance Tests on Blake

Introduction

Currently testing the Greenland Ice Sheet (GIS) and Antartica Ice Sheet (AIS) in Albany Land Ice (ALI) using Intel Skylake CPUs on blake.

Architectures:

Name Blake (SKX)
CPU Dual-socket Intel
Xeon Platinum 8160
(Skylake)
Cores/Node 48
Threads/Core 2
Memory/Node 187 GB
Interconnect Intel OmniPath
Gen-1 (100 GB/s)
Compiler Intel 18.1.163
MPI openmpi 2.1.2

Cases:

Case Name Number of Processes (np) Description
ant-2-20km_ml_ls 384 Unstructured 2-20km AIS, ML w/ line smoothing
ant-2-20km_mu_ls 384 Unstructured 2-20km AIS, MueLu w/ line smoothing
ant-2-20km_mu_dls 384 Unstructured 2-20km AIS, MueLu w/ decoupled line smoothing
greenland-1-7km_fea_1ws 384 Unstructured 1-7km GIS, finite element assembly only, single workset
greenland-1-7km_ml_ls_1ws 384 Unstructured 1-7km GIS, ML w/ line smoothing, single workset
greenland-1-7km_mu_ls_1ws 384 Unstructured 1-7km GIS, MueLu w/ line smoothing, single workset
greenland-1-7km_mu_dls_1ws 384 Unstructured 1-7km GIS, MueLu w/ decoupled line smoothing, single workset
greenland-1-7km_fea_mem 384 Unstructured 1-7km GIS, finite element assembly only, memoization
greenland-1-7km_ml_ls_mem 384 Unstructured 1-7km GIS, ML w/ line smoothing, memoization
greenland-1-7km_mu_ls_mem 384 Unstructured 1-7km GIS, MueLu w/ line smoothing, memoization
greenland-1-7km_mu_dls_mem 384 Unstructured 1-7km GIS, MueLu w/ decoupled line smoothing, memoization

Timers:

Timer Name Level Description
Albany Total Time 0 Total wall-clock time of simulation
Albany: Setup Time 1 Preprocess
Albany: Total Fill Time 1 Finite element assembly
Albany Fill: Residual 2 Residual assembly
Albany Residual Fill: Evaluate 3 Compute the residual, local/global assembly
Albany Residual Fill: Export 3 Update global residual across MPI ranks
Albany Fill: Jacobian 2 Jacobian assembly
Albany Jacobian Fill: Evaluate 3 Compute the Jacobian, local/global assembly
Albany Jacobian Fill: Export 3 Update global Jacobian across MPI ranks
NOX Total Preconditioner Construction 1 Construct Preconditioner
NOX Total Linear Solve 1 Linear Solve
In [1]:
import datetime as dt
import glob
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
import os

# Import scripts
from json2status import json2status
from json2timeline import json2timeline

# Extract file names
files = glob.glob('ctest-*')

Performance Tests Status

In [2]:
%matplotlib inline
#plt.rc('text', usetex=True)
plt.rc('font', family='serif', size=24)
plt.rcParams['figure.figsize'] = (14, 6)

# Cases to plot
cases = ('ant-2-20km_ml_ls',
         'ant-2-20km_mu_ls',
         'ant-2-20km_mu_dls',
         'green-1-7km_fea_1ws',
         'green-1-7km_ml_ls_1ws',
         'green-1-7km_mu_ls_1ws',
         'green-1-7km_mu_dls_1ws',
         'green-1-7km_fea_mem',
         'green-1-7km_ml_ls_mem',
         'green-1-7km_mu_ls_mem',
         'green-1-7km_mu_dls_mem')
np = 384

# Plot:
plt.figure()
imarker = 0
markers = ['+','o','*','x','v','d','^','s','>','<','+']
legNames = []
for case in cases:
    # Extract info and plot
    dates, status = json2status(files, case, np)
    fdates = [dt.datetime.strptime(str(d),'%Y%m%d').date() for d in dates]
    plt.plot(fdates, status, markersize=8.0, marker=markers[imarker], linestyle='None')
    imarker = imarker + 1
    legNames.append(case+'-np'+str(np))
        
plt.xlabel('Date')
plt.ylabel('Pass/Fail')
#plt.ylim((6e-3, 1e-1))
plt.legend(legNames,fontsize=14,loc='center left')
plt.tight_layout()
plt.title('Performance Tests Status')
ax = plt.gca()
ax.set_axisbelow(True)
ax.grid(linestyle='--', alpha=0.5)
ax.xaxis.set_major_locator(plt.MaxNLocator(5))
#ax.tick_params(axis='x',rotation = 45)
plt.show()

Performance Timelines

In [3]:
%matplotlib inline
#plt.rc('text', usetex=True)
plt.rc('font', family='serif', size=24)
plt.rcParams['figure.figsize'] = (12, 6)

# Cases to plot:
cases = ('ant-2-20km_ml_ls',
         'ant-2-20km_mu_ls',
         'ant-2-20km_mu_dls',
         'green-1-7km_fea_1ws',
         'green-1-7km_ml_ls_1ws',
         'green-1-7km_mu_ls_1ws',
         'green-1-7km_mu_dls_1ws',
         'green-1-7km_fea_mem',
         'green-1-7km_ml_ls_mem',
         'green-1-7km_mu_ls_mem',
         'green-1-7km_mu_dls_mem')
np = 384
names =  ('Total Time',
          'Setup Time',
          'Total Fill Time',
          'Residual Fill',
          'Residual Fill Evaluate',
          'Residual Fill Export',
          'Jacobian Fill',
          'Jacobian Fill Evaluate',
          'Jacobian Fill Export',
          'NOX Total Preconditioner Construction',
          'NOX Total Linear Solve')
timers = ('Albany Total Time:',
          'Albany: Setup Time:',
          'Albany: Total Fill Time:',
          'Albany Fill: Residual:',
          'Albany Residual Fill: Evaluate:',
          'Albany Residual Fill: Export:',
          'Albany Fill: Jacobian:',
          'Albany Jacobian Fill: Evaluate:',
          'Albany Jacobian Fill: Export:',
          'NOX Total Preconditioner Construction:',
          'NOX Total Linear Solve:')

# Plot:
for case in cases:
    for name, timer in zip(names, timers):
        dates, wtimes = json2timeline(files, case, np, timer, False)
        fdates = [dt.datetime.strptime(str(d),'%Y%m%d').date() for d in dates]

        # Plot
        plt.figure()
        plt.plot(fdates, wtimes, linewidth=3.0)
        plt.xlabel('Date')
        plt.ylabel('Wall-clock time (s)')
        #plt.ylim((6e-3, 1e-1))
        plt.tight_layout()
        plt.title(case + ' timeline - ' + name)
        ax = plt.gca()
        ax.set_axisbelow(True)
        ax.grid(linestyle='--', alpha=0.5)
        ax.xaxis.set_major_locator(plt.MaxNLocator(5))
        #ax.tick_params(axis='x',rotation = 45)
        plt.show()

        # Print
        print("Dates: ", dates)
        print("Wall-clock Time: ", wtimes)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200111)
Wall-clock Time:  (185.274, 201.549, 197.528, 208.615, 183.471, 194.819, 188.244, 190.846, 182.686, 197.163)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200111)
Wall-clock Time:  (5.51616, 13.1704, 15.1645, 7.28887, 6.57317, 14.5286, 8.51446, 9.28736, 6.60224, 12.418)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200111)
Wall-clock Time:  (154.922, 162.603, 156.099, 171.139, 148.456, 151.298, 154.881, 152.752, 151.872, 154.288)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200111)
Wall-clock Time:  (77.2019, 78.9798, 76.8354, 78.4598, 75.7741, 75.6764, 76.3313, 78.0529, 76.8571, 75.378)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200111)
Wall-clock Time:  (69.8607, 69.5994, 69.8973, 70.1796, 69.7059, 69.7783, 69.776, 69.9341, 69.9375, 70.2071)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200111)
Wall-clock Time:  (5.70978, 7.13628, 5.20056, 6.50826, 4.91687, 4.53196, 5.73409, 5.62516, 6.14941, 3.79208)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200111)
Wall-clock Time:  (77.7189, 83.6217, 79.2629, 92.6775, 72.6808, 75.6203, 78.5487, 74.6976, 75.0132, 78.9091)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200111)
Wall-clock Time:  (65.3743, 65.3251, 65.6604, 67.8577, 65.5838, 65.6251, 65.5711, 65.5708, 65.5009, 65.9551)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200111)
Wall-clock Time:  (4.82185, 6.92943, 4.48212, 15.1575, 5.06582, 5.75597, 5.91822, 4.67299, 4.56223, 5.55555)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200111)
Wall-clock Time:  (3.7695, 4.49467, 5.26111, 4.77997, 3.56072, 3.79174, 3.67986, 3.87113, 3.52729, 4.84835)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200111)
Wall-clock Time:  (20.3028, 20.1725, 19.7391, 24.1758, 24.0537, 24.1636, 19.8109, 23.8589, 20.0116, 24.3384)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (184.061, 188.156, 180.698, 187.553, 186.11, 186.3, 188.951, 185.356, 182.436, 195.772, 186.152)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (6.45611, 6.92513, 6.4935, 8.17776, 5.98308, 5.84309, 7.17387, 5.6664, 5.99887, 15.1506, 7.509)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (144.079, 145.316, 143.485, 145.453, 144.747, 144.389, 146.299, 144.402, 143.717, 149.287, 144.63)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (72.1694, 72.2507, 72.1649, 72.6369, 72.4191, 72.3791, 72.5243, 72.4152, 71.9354, 72.0762, 72.5392)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (69.7416, 69.6059, 69.8364, 69.8247, 69.6003, 69.6507, 69.7792, 69.8273, 69.835, 69.9288, 70.2816)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.888325, 0.982857, 0.841178, 1.10815, 1.03588, 1.06337, 1.02948, 1.05637, 0.683434, 0.772218, 0.785633)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (71.9089, 73.0644, 71.3196, 72.815, 72.3275, 72.0088, 73.7741, 71.9856, 71.7805, 77.2097, 72.0901)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (65.4934, 65.5543, 65.7607, 65.579, 65.4541, 65.3846, 65.6124, 65.6218, 65.2821, 65.5485, 65.6907)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.70382, 2.60968, 2.47276, 2.75184, 2.72144, 2.731, 2.96719, 2.60259, 2.75993, 2.52445, 2.69322)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (5.48295, 5.44062, 4.9945, 5.39091, 5.25201, 5.87993, 5.61347, 5.30123, 5.06355, 5.57823, 5.4275)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (23.1494, 24.9198, 21.3875, 25.2828, 25.199, 25.3439, 25.0112, 25.2189, 23.1526, 20.2672, 24.6787)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (181.559, 187.162, 179.775, 184.786, 183.268, 183.561, 184.286, 184.564, 180.025, 180.403, 183.854)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (5.9123, 5.51392, 5.28225, 5.65319, 5.65517, 5.71798, 5.4665, 5.58814, 5.45062, 5.79448, 5.68791)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (144.176, 145.235, 143.101, 144.718, 144.386, 144.36, 144.494, 144.634, 144.285, 143.563, 144.281)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (72.2062, 72.2826, 71.9351, 72.269, 72.0428, 72.1965, 72.1963, 72.5995, 72.3185, 71.9042, 72.5087)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (69.6457, 69.5742, 69.8892, 69.6763, 69.5862, 69.7707, 69.7428, 69.8323, 69.9277, 69.9298, 70.2235)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.993452, 0.926048, 0.684304, 0.965064, 0.910986, 0.934948, 0.890398, 1.12095, 0.857183, 0.703679, 0.823832)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (71.9685, 72.9518, 71.1655, 72.4483, 72.342, 72.1626, 72.2969, 72.0334, 71.9659, 71.6575, 71.7718)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (65.4941, 65.5026, 65.5488, 65.4495, 65.4185, 65.5382, 65.5626, 65.6255, 65.4163, 65.4304, 65.7487)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.64387, 2.66735, 2.54493, 2.74355, 2.73719, 2.74097, 2.79966, 2.59046, 2.80833, 2.45193, 2.65192)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (4.12441, 4.32488, 3.83063, 4.34351, 4.33311, 4.36679, 4.40563, 4.31225, 3.98421, 4.17083, 4.36694)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (22.3325, 25.4595, 21.3531, 25.1385, 25.0575, 25.2879, 25.3739, 25.0303, 22.0631, 22.9551, 25.5596)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (37.2355, 38.1735, 38.8363, 36.1227, 36.065, 37.4648, 36.0527, 36.9047, 35.7928, 37.599, 38.2341)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.14768, 3.08317, 3.15644, 1.08146, 1.05103, 2.67378, 1.07159, 1.14416, 1.04528, 2.18083, 2.17288)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (33.0787, 33.2894, 33.5774, 33.0503, 33.0344, 32.9527, 33.1615, 33.7378, 32.9462, 33.3978, 33.8113)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (9.71839, 9.72102, 9.74507, 9.70016, 9.67973, 9.76637, 9.71006, 9.77325, 9.71799, 9.7745, 9.8523)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (8.96973, 8.95696, 9.00623, 8.96144, 8.95139, 9.0203, 8.96529, 9.00086, 8.98874, 9.02401, 9.07465)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.238863, 0.242812, 0.244703, 0.234944, 0.232426, 0.237873, 0.247955, 0.245341, 0.239325, 0.237549, 0.231428)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (23.3558, 23.5639, 23.8278, 23.3457, 23.3502, 23.1817, 23.4469, 23.9602, 23.2236, 23.6188, 23.9545)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (18.147, 18.1343, 18.2722, 18.1471, 18.1499, 18.2696, 18.3003, 18.3351, 18.2561, 18.3266, 18.4095)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (3.05192, 3.02101, 3.0379, 3.04744, 3.03797, 3.07896, 3.0892, 3.07091, 3.04519, 3.05039, 3.03279)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.00647128, 0.00679808, 0.00640373, 0.00660568, 0.00662839, 0.00662837, 0.00654892, 0.0060899, 0.00646953, 0.00612059, 0.00621657)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.567963, 0.56698, 0.596383, 0.528657, 0.531359, 0.562384, 0.538508, 0.538958, 0.509912, 0.55727, 0.580554)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (9.4115, 9.68225, 9.54199, 9.42728, 9.34254, 9.49711, 9.39029, 9.83355, 9.40661, 9.86512, 10.2297)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.0894, 1.07315, 1.06032, 1.11708, 1.05481, 1.0671, 1.0687, 1.12905, 1.1176, 1.1218, 1.1741)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (4.82709, 5.02645, 4.96793, 4.83757, 4.78591, 4.80744, 4.8232, 5.08256, 4.80797, 4.91122, 5.13716)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.85396, 1.92323, 1.86408, 1.86414, 1.83364, 1.84864, 1.83803, 1.9441, 1.83413, 1.8706, 2.00567)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.34255, 1.34326, 1.3488, 1.34216, 1.33914, 1.34918, 1.34305, 1.35014, 1.34706, 1.35158, 1.35993)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.24615, 0.210639, 0.257046, 0.255288, 0.244883, 0.222652, 0.228653, 0.234934, 0.224039, 0.24833, 0.258298)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.97245, 3.1026, 3.10316, 2.97264, 2.9516, 2.95811, 2.98451, 3.13788, 2.97317, 3.04001, 3.13081)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.00519, 2.00029, 2.01543, 2.00638, 2.00011, 2.01175, 2.02086, 2.02003, 2.0182, 2.01991, 2.02465)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.529096, 0.511693, 0.519979, 0.51208, 0.498123, 0.4967, 0.525636, 0.505195, 0.513009, 0.572344, 0.501306)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.2168, 1.23857, 1.20975, 1.19294, 1.22622, 1.34133, 1.23312, 1.18717, 1.21596, 1.30679, 1.27045)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.12095, 2.13483, 2.12766, 2.11656, 2.11276, 2.1116, 2.11227, 2.19463, 2.10505, 2.35455, 2.36411)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (10.5075, 10.8269, 10.8616, 10.5212, 10.5465, 10.4665, 10.5109, 10.8757, 10.5206, 10.9025, 11.1336)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.06148, 1.04303, 1.24136, 1.04013, 1.05439, 1.04143, 1.05342, 1.07464, 1.04387, 1.08391, 1.09469)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (4.57414, 4.73833, 4.72438, 4.58499, 4.58926, 4.5523, 4.57894, 4.77826, 4.58124, 4.61175, 4.8276)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.6446, 1.65695, 1.64818, 1.64067, 1.64112, 1.64746, 1.63983, 1.66297, 1.64897, 1.64052, 1.69519)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.33843, 1.33803, 1.34123, 1.33709, 1.33861, 1.33953, 1.33772, 1.34354, 1.34364, 1.34398, 1.35062)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.0356494, 0.037106, 0.0365914, 0.0359748, 0.0346129, 0.0368749, 0.0359683, 0.0373973, 0.0354071, 0.0328885, 0.0362289)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.92893, 3.08066, 3.07557, 2.94372, 2.94754, 2.90422, 2.93849, 3.11466, 2.93166, 2.97063, 3.13179)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.00266, 2.00157, 2.01784, 2.00107, 2.00095, 2.00513, 2.01637, 2.02155, 2.01747, 2.01901, 2.03444)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.340061, 0.33563, 0.332719, 0.339322, 0.335752, 0.339733, 0.342374, 0.338507, 0.338715, 0.334264, 0.334136)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.61708, 1.67556, 1.66648, 1.6338, 1.6259, 1.64253, 1.64158, 1.64215, 1.65523, 1.76509, 1.69852)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.92803, 2.94257, 2.85506, 2.92761, 2.94514, 2.91228, 2.91516, 2.91203, 2.92267, 3.11407, 2.97692)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (10.5265, 10.8118, 10.6247, 10.5777, 10.5292, 10.4239, 10.4897, 11.0879, 10.5062, 10.7116, 11.1709)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.09273, 1.06095, 1.0434, 1.04211, 1.04464, 1.03694, 1.05062, 1.08479, 1.06158, 1.0356, 1.10201)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (4.56296, 4.74122, 4.71613, 4.59101, 4.5845, 4.53546, 4.56503, 4.91872, 4.56784, 4.63673, 4.83109)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.64482, 1.65563, 1.64027, 1.63733, 1.63664, 1.64061, 1.63882, 1.65649, 1.64015, 1.64075, 1.69209)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.33889, 1.33747, 1.34287, 1.33644, 1.33851, 1.3369, 1.33868, 1.33963, 1.34102, 1.34288, 1.34938)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.0370016, 0.0374048, 0.0345136, 0.0354777, 0.0343313, 0.0355857, 0.034452, 0.0372132, 0.0346647, 0.0343635, 0.0351376)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.91753, 3.08497, 3.07525, 2.95309, 2.94725, 2.89425, 2.92559, 3.26162, 2.92708, 2.99538, 3.13838)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.00181, 2.00055, 2.0143, 1.99972, 2.00171, 2.00153, 2.01543, 2.02466, 2.01359, 2.01702, 2.03491)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.337046, 0.334795, 0.333262, 0.339212, 0.335655, 0.336679, 0.340515, 0.349751, 0.337654, 0.337154, 0.334973)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.56373, 1.59217, 1.58677, 1.61234, 1.57482, 1.58452, 1.57664, 1.59505, 1.56783, 1.63692, 1.64224)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.98387, 2.99535, 2.91528, 3.00643, 3.00056, 2.95348, 2.98625, 3.00179, 2.99083, 3.06832, 3.05286)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (18.4797, 18.9268, 18.5707, 18.5324, 18.6384, 18.3729, 18.6899, 19.3241, 18.594, 20.0707, 19.3622)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.02663, 1.02486, 1.02582, 1.05439, 1.04466, 1.02581, 1.04132, 1.07559, 1.0331, 1.10987, 1.09413)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (16.5866, 16.8959, 16.6498, 16.5821, 16.6969, 16.4741, 16.7374, 17.205, 16.6723, 17.8246, 17.1454)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.00506, 2.01571, 2.00736, 2.00572, 2.00745, 1.99629, 1.99653, 2.01677, 2.01538, 2.01475, 2.06588)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.6485, 1.64489, 1.65489, 1.64744, 1.65005, 1.64192, 1.63911, 1.64965, 1.65526, 1.65278, 1.66311)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.0372619, 0.0399879, 0.0379209, 0.0379271, 0.0387606, 0.0375575, 0.0392794, 0.0380478, 0.0382152, 0.0387565, 0.041085)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (14.5772, 14.8759, 14.6381, 14.5721, 14.6851, 14.4735, 14.7364, 15.184, 14.6526, 15.8056, 15.0752)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (10.7219, 10.7018, 10.7538, 10.7161, 10.7265, 10.6893, 10.8233, 10.8153, 10.8379, 10.8341, 10.8628)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.56882, 2.57492, 2.5912, 2.57871, 2.61257, 2.58759, 2.64498, 2.68457, 2.58319, 2.60653, 2.57469)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.00617211, 0.00609199, 0.00580541, 0.0061533, 0.00606035, 0.00625412, 0.00617741, 0.00587767, 0.00617644, 0.00626846, 0.00573557)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.513182, 0.531325, 0.501477, 0.505681, 0.519786, 0.502168, 0.522027, 0.545384, 0.521874, 0.538862, 0.576756)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (7.24233, 7.60622, 7.41558, 7.32738, 7.35093, 7.29042, 7.28865, 7.88596, 7.359, 7.76018, 7.9503)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.04461, 1.09917, 1.04276, 1.04248, 1.04767, 1.04409, 1.06676, 1.1962, 1.07862, 1.31424, 1.22112)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.70238, 2.95982, 2.80888, 2.74883, 2.76492, 2.69918, 2.70117, 3.04695, 2.73244, 2.80161, 3.01741)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.635903, 0.726924, 0.619778, 0.651086, 0.636205, 0.62943, 0.615317, 0.757494, 0.648648, 0.64713, 0.759052)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.313967, 0.315233, 0.315487, 0.314193, 0.314426, 0.314703, 0.313382, 0.315272, 0.316475, 0.316563, 0.317257)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.0494081, 0.0462343, 0.0490486, 0.0651267, 0.049267, 0.0547316, 0.0466124, 0.0697398, 0.0621634, 0.0687975, 0.0588976)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.0659, 2.2323, 2.18853, 2.09718, 2.12813, 2.06919, 2.08527, 2.28868, 2.08323, 2.15384, 2.25769)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.24704, 1.24718, 1.25228, 1.2468, 1.24981, 1.25009, 1.25764, 1.26567, 1.26157, 1.26537, 1.26459)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.380776, 0.390783, 0.379858, 0.394745, 0.424858, 0.381267, 0.389494, 0.407477, 0.373255, 0.44377, 0.384758)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.18278, 1.24648, 1.21566, 1.18834, 1.19888, 1.20691, 1.19712, 1.23098, 1.21034, 1.26958, 1.23517)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.15098, 2.09466, 2.17139, 2.18133, 2.17402, 2.17559, 2.16127, 2.15942, 2.17071, 2.21617, 2.18886)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (8.4271, 8.81543, 8.49814, 8.47157, 8.48282, 8.45467, 8.45581, 8.96177, 8.43017, 8.96076, 9.01875)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.05025, 1.07574, 1.04372, 1.03957, 1.08644, 1.03847, 1.0673, 1.07157, 1.04424, 1.08465, 1.09905)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.6013, 2.78985, 2.72704, 2.63984, 2.64193, 2.62013, 2.666, 2.87277, 2.63128, 2.6773, 2.85344)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.564413, 0.577091, 0.560237, 0.563897, 0.565647, 0.567285, 0.563043, 0.582854, 0.568593, 0.561598, 0.606948)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.309749, 0.309847, 0.311861, 0.309966, 0.311103, 0.311345, 0.310457, 0.313793, 0.312396, 0.312137, 0.31264)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.00901744, 0.00971811, 0.00847501, 0.00910264, 0.00836127, 0.00915254, 0.00852224, 0.0102567, 0.00878789, 0.00801786, 0.00978093)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.03629, 2.21218, 2.16621, 2.07535, 2.07569, 2.05225, 2.10235, 2.28932, 2.06208, 2.11511, 2.24577)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.24776, 1.24712, 1.25288, 1.24719, 1.2494, 1.25387, 1.25949, 1.26637, 1.26282, 1.26166, 1.26091)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.289731, 0.286662, 0.29184, 0.287371, 0.292343, 0.294144, 0.300705, 0.295332, 0.290738, 0.29046, 0.293241)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.61944, 1.69178, 1.63632, 1.6233, 1.6206, 1.64126, 1.61673, 1.70889, 1.64366, 1.81619, 1.70284)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.97174, 2.9217, 2.88673, 2.97484, 2.94453, 2.96555, 2.92253, 2.95773, 2.92214, 3.1653, 2.9562)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (8.43109, 8.76709, 8.48415, 8.45662, 8.48786, 8.37906, 8.41034, 9.05234, 8.43949, 8.85104, 9.18824)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.05418, 1.05112, 1.04339, 1.04921, 1.08134, 1.0359, 1.04266, 1.08647, 1.04377, 1.10195, 1.11177)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.60875, 2.77934, 2.7111, 2.62075, 2.65096, 2.58979, 2.62013, 2.8605, 2.63577, 2.65603, 2.85035)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.564324, 0.576142, 0.556643, 0.561527, 0.564135, 0.55885, 0.55863, 0.577411, 0.568616, 0.562282, 0.613363)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.310135, 0.310119, 0.309744, 0.309065, 0.31161, 0.308255, 0.308387, 0.311974, 0.311979, 0.312944, 0.311016)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.00884691, 0.00969436, 0.00800962, 0.00886958, 0.00850671, 0.00891768, 0.00889261, 0.0105084, 0.00935285, 0.007898, 0.0127749)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (2.04383, 2.2026, 2.15386, 2.05864, 2.08623, 2.03036, 2.0609, 2.28249, 2.06655, 2.09316, 2.23636)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.24859, 1.24664, 1.25164, 1.24706, 1.25011, 1.24655, 1.25781, 1.26697, 1.26241, 1.26198, 1.26328)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (0.288858, 0.2884, 0.287396, 0.28853, 0.292156, 0.290218, 0.299965, 0.299442, 0.290257, 0.28898, 0.292625)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (1.57008, 1.64844, 1.58302, 1.58002, 1.55719, 1.58573, 1.57037, 1.70425, 1.58487, 1.69172, 1.75637)
Dates:  (20191227, 20191228, 20191229, 20191231, 20200101, 20200102, 20200107, 20200108, 20200109, 20200110, 20200111)
Wall-clock Time:  (3.01777, 2.97498, 2.95306, 3.00487, 3.00961, 2.98587, 2.99823, 3.02008, 2.98637, 3.17934, 3.05525)